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SECTION  I 
INTRODUCTION 

The  training  of  maintenance  skills  is  an  issue  of  major  concern 
to  the  military  due  to  increases  in  the  complexity  of  in-the-field 
hardware  and  the  cost  of  training  maintenance  technicians.  Effective 
maintenance  training  programs  can  facilitate  full  utilization  of 
hardware  technology  and  decrease  the  maintenance  costs  by  reducing  the 
number  of  technicians  required,  spare  parts  costs,  and  hardware  down 
time.  Traditionally,  hands-on  electronic  maintenance  training  has 
utilized  actual  equipment  trainers  (AETs)  to  provide  troubleshooting 
training  to  the  component  level.  The  primary  objective  of  this  train¬ 
ing  has  been  to  develop  and  sustain  applied  skills  and  knowledge  in 
basic  electricity  and  electronics.  The  Basic  Electricity  and 
Electronics  (BE&E)  School,  located  at  the  Naval  Training  Center  in 
Orlando,  Florida,  utilizes  a  computer  managed  course  of  instruction 
that  employs  hands-on  maintenance  training  in  conjunction  with  self- 
paced  instructional  texts  to  train  basic  troubleshooting  concepts  and 
skills . 

A  recent  study  conducted  by  McDonald  &  Associates,  Inc. 
(McDonald,  Waldrop,  &  White,  1982)  at  the  BE&E  School  revealed  that 
students  did  not  always  utilize  optimum  troubleshooting  procedures. 
Efficient  electronic  troubleshooting  requires  isolation  of  the  faulty 
component  by  taking  readings  at  logical  test  points  and  using  the 
information  from  those  readings  to  determine  the  next  logical  test 
point.  A  review  of  the  School's  curriculum  indicated  that  logical 
troubleshooting  behavior  was  addressed,  but  not  stressed,  and  as  a 
result,  students  adopted  many  different  combinations  of  troubleshoot¬ 
ing  strategy  in  actual  hands-on  performance  tests.  The  combinations 
adopted  did  not  always  lead  to  optimum  troubleshooting  performance  and 
many  times  resulted  in  inefficient  test  point  probes. 

The  use  of  computer  assisted  instruction  (CAI)  that  interfaces 
simultaneously  with  an  interactive  video  can  optimize  the  costs  and 
time  associated  with  maintenance  skills  training.  Many  of  the  ben¬ 
efits  derived  from  CAI  are  directly  applicable  to  a  maintenance  train¬ 
ing  program  like  the  one  currently  used  at  BE&E.  CAI  can  provide 
practices,  reviews,  and  performance  tests  directly  to  the  student, 
freeing  the  Learning  Supervisor  (LS)  to  perform  other  instructional 
functions.  Student  performance  can  also  be  timed,  evaluated,  and 
remediated  by  the  computer.  Computer  models,  simulating  actual  equip¬ 
ment,  reduce  the  number  of  AETs  and  circuit  boards  needed,  thus 
reducing  equipment  costs.  Test  point  probing  and  assignment  of  faulty 
circuits  are  done  by  computer,  eliminating  deadtime  due  to  the  servic¬ 
ing  of  damaged  actual  equipment  and  circuit  boards.  In  addition,  CAI 
allows  the  constant  up-date  of  instructional  information  without 
costly  text  rewrites.  CAI  has  the  potential  to  provide  optimum 
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individualized  maintenance  training,  as  well  as  to  cut  costs  by  manag¬ 
ing  the  course  curriculum. 

To  test  the  applicability  of  CAI  to  the  BE&E  School,  a  pilot 
study  was  conducted  using  an  off-the-shelf  computer/ video  assisted 
instructional  program  on  strategic  troubleshooting.  The  course 
emphasized  taking  a  logical  sequence  of  tests,  based  on  both  good  and 
bad  inputs  and  outputs,  to  localize  the  faulty  component  with  the 
least  number  of  testprobes.  Twelve  students  participated  in  the  study 
(four  experimental  and  eight  controls).  Each  experimental  subject  was 
matched  with  a  male  and  female  control  subject.  Results  from  the 
study  indicated  that  the  experimental  subjects  spent  less  time  trouble¬ 
shooting  a  faulty  board,  took  fewer  probes,  and  spent  less  time  com¬ 
pleting  the  BE&E  Electronic  Technician  (ET)  curriculum.  Results  from 
the  pilot  study  were  encouraging  but  difficult  to  generalize  to  the 
overall  population,  due  to  the  small  sample  size.  In  order  to  deter¬ 
mine  whether  or  not  the  course  was  effective,  further  research  was 
required  utilizing  a  larger  sample  size,  with  appropriate  control 
conditions . 

Methods  similar  to  those  in  the  pilot  study  were  used  in  the 
current  research  to  implement  the  course  just  prior  to  students  enter¬ 
ing  the  ET  Splice  modules  of  BE&E  School.  The  purpose  of  the  study 
was  to  examine  the  effectiveness  of  the  troubleshooting  CAI  course  on 
troubleshooting  behavior  during  performance  tests  in  the  ET  Splice 
phase  of  instruction.  The  hypotheses  to  be  tested  were: 

a.  Students  participating  in  the  troubleshooting  CAI  will  trou¬ 
bleshoot  more  efficiently  than  control  students. 

b.  High  proficiency  students  will  troubleshoot  more  efficiently 
than  medium  and  low  proficiency  students,  and  medium  profi¬ 
ciency  students  will  perform  more  efficiently  than  low  profi¬ 
ciency  students. 

c.  High  proficiency  students  will  complete  the  ET  Splice  curric¬ 
ulum  in  fewer  hours  than  medium  and  low  proficiency  students, 
and  medium  proficiency  students  will  complete  the  ET  Splice 
curriculum  in  fewer  hours  than  low  proficiency  students. 

d.  Students  participating  in  the  troubleshooting  CAI  will  com¬ 
plete  the  ET  Splice  curriculum  in  fewer  hours  than  control 
students . 
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SECTION  II 
METHOD 

The  primary  objective  of  the  research  program  was  to  determine 
the  effects  of  the  strategic  troubleshooting  course  on  student  trouble¬ 
shooting  efficiency  on  a  typical  troubleshooting  task  of  the  ET  Splice 
course.  Since  the  purpose  of  the  study  was  to  determine  the  transfer 
of  training  of  the  troubleshooting  CAI  to  actual  hands-on  perfor¬ 
mance,  the  research  was  designed  to  be  integrated  into  the  testing 
procedures  used  at  the  BE&E  School.  The  CAI  workbook  and  program  were 
extensively  modified  to  present  the  student  with  curriculum  similar  to 
that  used  at  the  BE&E  School  and  to  fulfill  research  data  require¬ 
ments.  The  BE&E  School  was  used  as  a  model  since  the  results  of  this 
study  were  intended  for  use  by  curriculum  designers  of  military  elec¬ 
tronics  training  with  additional  application  to  industrial  training 
possible . 

APPARATUS 

STRATEGIC  TROUBLESHOOTING  COURSEWARE.  The  experimental  treatment 
consisted  of  an  off-the-shelf  strategic  troubleshooting  course.  This 
course  combined  videotape  presentations,  workbook  exercises,  and 
computer-assisted  instruction  (CAI)  materials.  The  computer  graphical¬ 
ly  presented  hypothetical  circuits  with  bad  outputs  and  allowed  the 
student  to  select  test  points  and  see  the  results  of  the  tests.  The 
computer  provided  feedback  on  whether  or  not  a  proper  troubleshooting 
strategy  was  being  used.  The  principal  troubleshooting  strategy 
taught  by  this  course  is  the  half-split  technique,  which  involves 
successive  testing  of  the  midpoint  between  known  good  and  bad  signals 
until  the  fault  is  isolated.  The  program  presentation  time  was  a 
minimum  of  9  hours,  and  additional  time  was  required  when  students 
repeated  units,  reviewed  practice  problems,  or  required  additional 
c lar if icat ion. 

BASIC  COURSEWARE.  A  CAl/video  program  which  taught  the  BASIC  computer 
language  was  developed  for  use  with  the  control  subjects.  The  course 
was  designed  to  be  similar  in  length  and  instructional  characteristics 
to  the  troubleshooting  CAI,  in  order  to  make  the  course  appear  rel¬ 
evant  to  the  control  subjects,  while  avoiding  any  material  which  might 
be  directly  applicable  to  troubleshooting.  The  video  portion  entitled 
"Computer  Programming:  BASIC  for  Microcomputers"  was  made  available 
from  Educational  Activities,  Inc.  and  was  integrated  with  a  TRS-80 
Model  III  BASIC  computer  interactive  course. 

CAI  EQUIPMENT.  Two  TRS-80  Model  III  computers  and  2  Betamax  video 
playback  units  with  video  monitors  were  used  to  present  the  trouble¬ 
shooting  CAI.  Headphones  were  utilized  to  prevent  interference  during 
simultaneous  operation  of  the  two  stations.  The  treatment  control  CAI 
(BASIC  course)  utilised  the  same  equipment  as  the  troubleshooting  CAI. 
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1  KoiiiM.i-.SiiOU  [' ING  EQUIPMENT.  Since  complexity  ol  the  troubleshooting 
task,  was  certain  to  affect  student  behavior,  3  different  printed 
circuit  boards  were  utilized  for  the  collection  of  actual  troubleshoot¬ 
ing  performance  data:  a  simple  205-5  Second  IF  Amplifier  (Second  IF), 
a  medium  complexity  205-4  First  IF  Amplifier  (First  IF)  and  a  highly 
complex  Power  Supply  (Power  Supply)  board  with  feedback  loops.  These 
boards  were  contained  in  a  NIDA  Model  205  Transceiver  Trainer  and  a 
NIDA  Model  201  Power  Supply  Trainer  utilized  as  a  normal  part  of  the 
curriculum  in  the  ET  Splice  course. 

The  study  utilized  9  prefaulted  boards  for  each  of  the  3  printed 
circuit  board  types,  providing  27  prefaulted  boards.  Boards  were 
pcefaulted  by  Che  manufacturer.  The  9  faulted  boards  for  each  board 
type  were  divided  into  3  fault  groups,  based  on  fault  difficulty. 
This  allowed  random  assignment  of  faults  to  each  student,  to  prevent 
the  possibility  of  prior  student  knowledge  of  fault  location  and  to 
reduce  performance  variance  due  to  fault  difficulty  differences. 

A  total  of  4  trainers  (2  201  Power  Supply  Trainers  and  2  205 
Transceivers)  were  available,  thus  allowing  any  combination  of  2  sep¬ 
arate  performance  tests  to  be  observed  at  one  time.  Additional  trou¬ 
bleshooting  equipment  included:  2  sweep  generators,  2  oscilloscopes, 

2  Simpson  Multimeters,  and  various  probes.  Any  additional  equipment 
required  was  supplied  by  the  School,  and  equipment  and  circuit  boards 
were  maintained  by  the  School. 

SUBJECTS 

Subjects  were  selected  from  students  enrolled  in  the  ET  Splice 
program  Modules  30  to  34  at  the  BE&E  School.  These  modules  are  a 
preparatory  course  for  an  Electronic  Technician  rating.  Students  were 
male,  E3  Seamen  ranging  in  age  from  17  to  35  with  the  average  age 
being  19.  The  education  level  ranged  from  completion  of  high  school 
to  1  year  of  college. 

All  students  were  tracked  prior  to  entering  ET  Splice  using  the 
School's  computer  managed  instruction  (CMI)  printouts.  This  allowed 
the  researcher  to  predict  when  students  would  be  entering  ET  Splice 
and  ready  to  be  assigned  to  1  of  the  3  treatment  conditions. 

Each  student  was  assigned  a  proficiency  level  of  high,  medium  or 
low  based  on  their  actual  elapsed  time  listed  on  the  daily  CMI  print¬ 
outs.  This  time  represents  a  student's  total  contact  time,  accumulat¬ 
ed  from  the  time  the  student  entered  the  BE&E  curriculum  to  just  prior 
to  entering  ET  Splice. 
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Proficiency  levels  were  hypothesized  to  significantly  influence 
troubleshooting  behavior.  Student  proficiency  categories  had  been 
determined  during  previous  research  (McDonald,  et.al.,  1982)  by  look¬ 
ing  ac  a  random  sample  of  225  student  BE&E  School  completion  times. 
Proficiency  categories  were  determined  by  monitoring  completion  times 
on  BE&E  CMI  printouts  and  dividing  the  range  of  times  into  3  equal 
groups  of  75  each.  This  resulted  in  the  following  proficiency  levels: 
high  proficiency  0-224.99  hours,  medium  225-289.99  hours,  and  low 
290-365.99  hours.  However,  between  the  previous  research  and  the 
current  research,  changes  in  the  School's  curriculum,  physical  lay¬ 
out  ,  and  student  population  resulted  in  a  change  in  the  time  distribu¬ 
tion  of  student  hours.  Using  the  same  process  and  the  existing 
categories  as  a  base,  new  BE&E  completion  times  were  monitored,  and 
the  resulting  categories  used  to  reclassify  student  proficiency  were: 
high  less  Chan  212  hours,  medium  212-311.99  hours,  and  low  312-411.99 
hours . 

ET  Splice  students  were  randomly  selected  after  being  tracked 
through  the  CMI  data  and  classified  in  1  of  the  3  proficiency  levels. 
Six  students  from  each  of  the  3  proficiency  levels  were  assigned  to  1 
of  the  3  treatment  conditions.  A  total  of  54  subjects  were  used  in 
repeated  measures  across  all  3  circuit  board  types  (18  troubleshooting 
CAI ,  18  control  CAI,  and  18  no-treatment  controls). 

Fifty-four  students  took  a  total  of  162  performance  tests  across 
all  3  boards.  A  total  of  54  performance  tests  (6  at  each  of  the  3 
proficiency  levels  on  each  of  the  3  types  of  boards)  were  observed  for 
each  of  the  treatment  conditions.  Experimental  matrices  were  used  to 
assure  that  all  treatment  conditions  were  balanced  and  completely 
randomized.  Student  attrition  occurred  periodically  due  to  equipment 
malfunction,  student  double-shifting  because  of  transfer  orders,  or 
reclassification  out  of  the  ET  Splice  curriculum  into  an  Electronics 
Warfare  program.  This  had  no  effect  on  finishing  the  complete  repeat¬ 
ed  measures  since  these  students  were  replaced  by  others  during  the 
time  set  aside  for  data  collection. 

PROCEDURE 

The  equipment  for  the  presentation  of  the  troubleshooting  CAI  and 
control  CAI  conditions  was  set  up  at  the  Orlando  Naval  Training 
Equipment  Center's  Human  Factors  Laboratory.  Two  stations  were  avail¬ 
able  with  only  1  type  of  treatment  condition  run  at  one  time,  i.e.,  2 
treatment  control  students  ^r  2  experimental  treatment  students. 
Headphones  were  used  to  prevent  the  2  stations  from  interfering  with 
one  another.  A  researcher  was  provided  by  the  Human  Factors  Laboratory 
to  monitor  the  2  CAI  training  stations  and  to  administer  tests.  Two 
stations  were  also  set  up  at  the  BE&E  School  for  collection  of  criteri¬ 
on  performance  data. 
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A  pilot  subject  was  sent  to  the  Human  Factors  Laboratory  to 
participate  in  the  troubleshooting  CAI.  After  completing  the  course, 
the  student  returned  to  the  BE&E  School  and  was  observed  on  1  Power 
Supply  and  2  IF  Amplifier  performance  tests.  This  preliminary  data 
provided  the  on-site  researcher  with  information  on  the  CAI  program 
ciming,  student  acceptance  of  the  CAI,  and  any  possible  BE&E  curric¬ 
ulum  interference.  In  addition,  it  allowed  the  researcher  to  begin 
tracking  students  at  BE&E  School,  to  assign  proficiency  levels  and 
treatment  conditions. 

Students  assigned  to  the  troubleshooting  CAI  or  control  CAI  were 
sent  to  the  Human  Factors  Laboratory  and  told  to  report  there  for  the 
next  few  days  instead  of  reporting  to  BE&E  School.  These  students 
were  put  on  temporary  hold  on  the  School's  CMI  system  so  the  2  to  3 
class  days  spent  partic ipating  in  the  CAI  treatment  condition  would 
not  affect  their  class  standing.  After  students  completed  their 
assigned  CAI  condition,  they  returned  to  BE&E  and  proceeded  with  their 
normal  ET  Splice  curriculum.  The  LS  sent  all  BE&E  students  to  the 
research  station  when  they  were  ready  for  performance  tests  on  Module 
30-2  (Power  Supply)  or  Module  31-3  (Transceiver).  This  allowed  the 
researcher  to  observe  performance  tests  from  the  students  who  partici¬ 
pated  in  the  CAI  courses,  as  well  as  to  randomly  select  no-treatment 
control  students.  Eighteen  troubleshooting  CAI  students,  18  control 
CAI,  and  18  no-treatment  control  students  were  observed  at  the 
researcher's  station.  Three  performance  tests  were  observed  for  every 
student,  1  on  the  Power  Supply,  1  on  the  First  IF  Amplifier  and  1  on 
the  Second  IF  Amplifier,  totalling  162  performance  tests.  A  pre¬ 
faulted  circuit  board  was  randomly  selected  from  the  appropriate  fault 
difficulty  group  for  each  student  by  the  researcher  before  each  perfor¬ 
mance  test. 

The  ET  Splice  curriculum  is  a  self-paced  program.  Students 
participating  in  the  research  program  took  their  performance  tests  in 
normal  sequence,  without  affecting  their  normal  course  workload  or 
hours.  The  only  modification  was  that  3  of  their  performance  tests 
were  taken  at  the  research  station  using  a  circuit  board  which  was 
assigned  by  the  researcher,  rather  than  by  an  LS.  The  ET  Splice 
curriculum  utilizes  3  different  trainers,  administering  7  practice 
exercises  and  7  performance  tests,  on  7  different  printed  circuit 
boards.  The  research  data  collected  represents  3  performance  tests  on 
2  of  the  3  trainers.  The  average  ET  Splice  completion  time  is  60 
classroom  hours,  and  the  typical  class  day  runs  6  hours. 

Before  taking  performance  tests,  students  were  briefed  and 
assured  that  data  collected  would  not  affect  their  class  standing. 
Students  used  the  School's  Troubleshooting  Performance  Response  Sheets 
when  taking  the  performance  tests.  The  student  informed  the  research¬ 
er  when  the  fault  was  diagnosed.  The  student  then  took  the  response 
sheet  to  the  LS  for  feedback  on  whether  or  not  the  diagnosis  was 
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correct.  If  incorrect,  the  student  returned  to  the  research  station 
to  continue  troubleshooting  the  same  faulty  circuit  until  the  correct 
fault  was  diagnosed.  After  the  correct  fault  was  diagnosed,  the 
student  returned  the  completed  response  sheet  to  the  researcher.  If 
the  performance  test  was  on  the  Power  Supply,  the  students  returned  to 
the  regular  BE&E  curriculum  until  Module  31-3  when  they  were  again 
referred  by  the  LS  to  the  research  station.  Prefaulted  circuit  boards 
on  the  NIDA  205  trainer  were  issued  in  random  sequence;  thus,  students 
could  receive  either  a  faulty  First  IF  Amplifier  or  a  faulty  Second  IF 
Amplifier  as  their  first  205  trainer  fault  card  and  could  receive  the 
remaining  one  as  their  second  performance  test  measure.  Students 
filled  out  response  sheets  for  every  performance  test  taken  and  again 
went  to  the  LS  for  feedback  on  their  fault  diagnosis.  All  response 
sheets  were  returned  to  the  researcher  after  the  correct  fault  was 
diagnosed. 

After  students  completed  all  3  performance  tests  at  the  research¬ 
er's  testing  area,  their  daily  progress  at  the  School  was  monitored  on 
the  CMI  to  obtain  student  response  histories  after  they  completed  the 
final  BE&E  School  test.  Student  response  histories  provided  the 
researcher  with  each  student's  total  BE&E  School  completion  time. 

During  the  3  performance  tests  taken  at  the  research  station  by 
each  of  the  54  students,  the  researcher  recorded  the  dependent  perfor¬ 
mance  measures  of:  specific  test  points  taken,  total  number  of 
probes,  total  probe  time,  fault  diagnosed,  student  comments,  number  of 
crips  to  the  LS  and  any  additional  relevant  data.  The  162  performance 
tests  represent  the  criterion  measure  to  determine  any  transfer  of 
training  effects  from  the  experimental  treatment  to  actual  hands-on 
performance  tests  and  any  effect  on  troubleshooting  behavior  due  to 
proficiency  level.  In  addition,  each  student's  overall  BE&E  course 
completion  time  was  analyzed  to  determine  any  effect  due  to  treatment 
or  proficiency  level.  Data  from  these  analyses  are  discussed  in 
detail  in  the  Results  section. 

EXPERIMENTAL  DESIGN 

The  two-dimensional  design  matrix  was  a  3  (Treatment  Condition) 
by  3  (Proficiency  Level)  design  with  3  replications  across  circuit 
boards.  The  main  independent  variables  were  Che  treatment  conditions 
(experimental  treatment,  treatment  control  and  baseline  control). 
Proficiency  level  was  used  as  a  blocking  variable,  assigning  subjects 
to  1  of  3  predetermined  categories  (blocks)  according  to  each  individ¬ 
ual's  total  number  of  hours  in  the  BE&E  curriculum  prior  to  ET  Splice. 
Assigning  a  specific  number  of  subjects  from  each  proficiency  level  to 
each  treatment  removes  a  large  source  of  potential  variation  that 
might  have  occurred  had  proficiency  levels  been  assigned  randomly  to 
treatment  conditions.  Fault  groups  and  proficiency  levels  were 
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matched  across  all  3  treatment  conditions  to  diminish  the  effects  of 
extraneous  variance  and  control  for  individual  differences. 

The  experimental  design  studied  the  effects  of  the  troubleshoot¬ 
ing  CAI  course  versus  two  types  of  control  groups.  The  treatment 
control  group  was  employed  to  account  for  any  Hawthorne  effect  on 
student  performance  which  might  result  from  changes  in  the  student's 
normal  instructional  environment.  The  treatment  control  manipulated 
the  student's  environment  and  the  sequence  of  learning  so  that  stim¬ 
ulus  conditions  closely  corresponded  to  those  of  the  experimental 
treatment  group  (troubleshooting  CAI).  The  baseline  (no- treatment) 
control  group  was  used  to  compare  the  effects  of  the  troubleshooting 
CAI  against  the  regular  class  curriculum,  as  well  as  to  examine  the 
possible  Hawthorne  effects  of  removing  the  student  from  the  BE&E 
environment.  The  matrix  represents  an  independent  design  in  that  a 
group  of  students  experience  only  one  treatment  condition  across  all  3 
boards.  The  experimental  design  is  represented  in  Figure  1. 

The  dependent  variables  under  study  were  probe  time,  number  of 
test  points  probed,  correctness  on  first  fault  diagnosis,  and  total 
number  of  hours  to  finish  BE&E  School.  In  addition  to  analysis  on  the 
main  dependent  variables,  the  troubleshooting  logic  used  by  students 
was  examined.  All  data  were  collected  on  3  performance  tests  taken  by 
each  student  on  the  3  circuit  boards.  Analysis  of  these  data  indicat¬ 
ed  the  degree  of  transfer  of  training  from  the  treatment  conditions. 

A  two-way  Analysis  of  Variance  (ANOVA)  procedure  was  used  to 
analyze  the  main  effects  of  the  design  matrix  (Ferguson,  1976),  repli¬ 
cated  across  the  3  circuit  board  types.  This  analysis  allowed  the 
simultaneous  examination  of  both  the  independent  and  combined  effects 
of  treatment  conditions  and  proficiency  levels.  Each  student  was 
classified  in  1  proficiency  level  and  exposed  to  1  set  of  experimental 
conditions  across  all  3  boards.  However,  since  the  effect  of  differ¬ 
ences  between  the  3  board  types  was  not  a  primary  research  question, 
the  boards  were  analyzed  as  independent  designs.  The  2  CAI  treatment 
conditions  (experimental  and  control)  were  examined  for  any  variabil¬ 
ity  in  troubleshooting  performance  when  compared  to  the  no-treatment 
control  group. 

Dichotomous  data  were  analyzed  using  a  Chi-Square  test  (Siegel, 
1956).  These  data  included  whether  or  not  the  student  was  correct  on 
the  first  fault  diagnosis  attempt  and  the  number  of  troubleshooting 
strategies  used  prior  to  the  first  fault  diagnosis.  This  analysis 
allowed  examination  of  the  frequency  of  correctness  on  first  fault 
diagnosis  and  number  of  strategies  used,  between  the  treatment  condi¬ 
tions.  In  addition,  Chi-Square  procedures  were  used  to  compare  suc¬ 
cess  of  Half-Split  troubleshooting  procedure  versus  all  other 
strategies  combined  on  the  first  fault  diagnosis. 
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PROFICIENCY  LEVEL 


TREATMENT  CONDITIONS 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

Control  (BASIC)  CAI 

No-Treatment  Control 

Figure  1.  Experimental  design  matrix. 
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Analysis  of  Variance  procedures  were  used  to  look  at  differences 
between  treatment  conditions.  The  level  for  significant  difference 
was  .05,  i.e.,  chere  must  be  a  95Z  probability  that  the  difference  is 
not  due  to  chance.  Analysis  of  Variance  procedures  only  indicated 
that  there  were  significant  differences  between  independent  variables. 
In  order  to  ascertain  where  the  significant  differences  were  occur¬ 
ring,  a  Fisher  Least  Significant  Difference  (LSD)  post  hoc  procedure 
(Wilkowitz,  et  al.,  1976)  was  performed  on  all  significant  ANOVA  f 
tests.  This  compared  all  possible  paired  means  using  the  Mean  Square 
Within  as  the  population  variance  estimate.  Again,  the  probability 
level  for  significance  was  .05. 
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SECTION  III 
RESULTS 

ANALYSIS  OF  VARIANCE  -  MAIN  EFFECTS 

The  results  discussed  in  this  section  examine  criterion  : erfor- 
mance  differences  between  students  in  the  experimental  treatment,  the 
control  treatment  and  the  no-treatraent  (baseline)  control  conditions. 
The  primary  measures  of  effectiveness  were  the  number  of  probes  and 
the  time  taken  to  locate  the  fault  during  fault  isolation  on  the 
criterion  boards.  The  experimental  and  control  conditions  were  exam¬ 
ined  within  the  3  board  types  and  analyzed  as  separate  ANOVA  designs. 

POWER  SUPPLY  BOARD.  The  ANOVA  totals  and  summary  data  for  number  of 

points  probed  are  shown  in  Tables  1  and  2.  These  data  indicate  a 

significant  (p<.05)  performance  difference  between  treatment  condi¬ 
tions.  Student  proficiency  level,  however,  did  not  significantly 
affect  the  number  of  points  probed. 

The  least  significant  difference  (LSD)  post  hoc  technique  was 
applied  to  determine  exactly  which  variable  differences  were  signif¬ 
icant.  The  LSD  uses  the  smallest  value  which  can  be  considered  signif¬ 
icant.  This  technique  examines  all  Che  pair-wise  mean  differences 
within  a  variable  (e.g.,  treatment  conditions)  to  determine  which 
difference  is  the  source  of  significance.  Table  3  contains  the  mean 
data  for  the  treatment  conditions.  The  LSD  indicated  a  significant 
(p<. 05)  difference  in  the  number  of  points  probed  between  Che  baseline 

control  group  and  the  troubleshooting  CAI  group  (experimental  treat¬ 

ment),  with  the  control  group  probing  fewer  points. 

The  ANOVA  totals  and  summary  data  contained  in  Tables  4  and  5 
indicated  a  significant  (p<.05)  effect  due  to  treatment  condition  on 
Che  time  taken  to  isolate  Che  fault  (probe  time  in  minutes).  Table  6 
contains  the  mean  probe  time  for  the  Power  Supply  board  by  treatment 
condition.  The  LSD  post  hoc  analysis  indicated  chat  the  baseline 
control  group  took  significantly  (p<.05)  less  time  to  locate  the  fault 
than  the  troubleshooting  CAI  group.  Student  proficiency  level  did  not 
significantly  affect  the  amount  of  time  required  to  isolate  the  fault. 

FIRST  IF  BOARD.  The  ANOVA  totals  and  summary  data  for  points  probed 
on  the  First  IF  board  are  contained  in  Tables-  7  and  8.  The  ANOVA 
results  indicated .  a  significant  (p<.05)  effect  due  to  student  profi¬ 
ciency  level.  Table  9  contains  the  mean  number  of  points  probed  by 
proficiency  level.  The  LSD  post  hoc  analysis  indicated  a  significant 
(p<.05)  difference  between  high  and  medium  proficiency  students  and 
between  high  and  low  proficiency  students  with  the  higher  proficiency 
subjects  probing  fewer  points.  The  treatment  conditions  did  not  have 
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TABLE  1 .  POWER  SUPPLY  BOARD  -  ANOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

183 

502 

349 

Control  CAI 

303 

173 

389 

Control 

161 

146 

144 

TABLE  2.  POWER  SUPPLY  BOARD  -  ANOVA  SUMMARY  -  NUMBER  OF  POINTS  PROBED 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Proficiency 

1652.18 

2 

826.09 

.54 

Treatment 

9997.06 

2 

4998.53 

3.29* 

Interaction 

10803.16 

^4 

2700.79 

1.78 

Error 

68310.90 

45 

1518.02 

Total 

90763.36 

53 

NOTE:  *  p<.05 

TABLE  3.  POWER  fUPPLY  BOARD  -  TREATMENT  CONDITION 
ISAM  POINTS  PROBED 


TREATMENT  CONDITION 

MEAN  POINTS  PROBED 

Troubleshooting  CAI 

57.44 

Control  CAI 

48.06 

Control 

25.06 
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TABLE  4.  POWER  SUPPLY  BOARD  -  ANOVA  TOTALS  - 
PROBE  TIME  (MINUTES) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

NIGH 

MEDIUM 

LOW 

4 

V 

Troubleshooting  CAI 

360 

374 

401 

> 

Control  CAI 

324 

265 

401 

* 

s. 

If 

? 

Control 

199 

183 

234 

i 

e, 

i 

TABLE 

5.  POWER  SUPPLY  BOARD 

-  ANOVA  SUMMARY  -  PROBE  TIME  (MINUTES) 

VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

\ 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE  F 

' 

Prof ic iency 

1350.52 

2 

675.26 

.64 

Treatment 

7967.82 

2 

3983.91 

3.78* 

Interaction 

571.36 

4 

142.84 

.14 

i 

Error 

47432.25 

45 

1054.05 

| 

Total 

57322.15 

53 

NOTE:  *p< .05 

TABLE  6.  POWER  SUPPLY  BOARD  -  TREATMENT  CONDITION 
MEAN  PROBE  TIME  (MINUTES) 


TREATMENT  CONDITION 

MEAN  TIME 

'  i 

Troubleshooting  CAI 

63.06 

Control  CAI 

55.00 

Control 

34.22 
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TABLE  7.  FIRST  IF  BOARD  -  ANOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

142 

289 

222 

Control  CAI 

143 

291 

285 

Control 

88 

156 

95 

TABLE  8. 

FIRST  IF  BOARD  -  ANOVA 

SUMMARY  - 

NUMBER  OF  POINTS 

PROBED 

VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE 

F 

Proficiency 

4466.00 

2 

2233.00 

3.72* 

Treatment 

2380.58 

2 

1190.29 

1.98 

Interaction 

656.00 

4 

164.00 

.27 

Error 

26984.70 

45 

599.66 

Total 

34487.63 

53 

NOTE:  *p< .05 

TABLE  9.  FIRST 

IF  BOARD 

-  PROFICIENCY- 

MEAN  POINTS  PROBED 


PROFICIENCY 

MEAN  POINTS  PROBED 

High 

20.72 

Medium 


40.89 
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a  significant  effect  on  the  number  of  points  probed  on  the  First  IF 
board. 

Tables  10  and  11  contain  the  ANOVA  totals  and  summary  data  for 
the  probing  time  on  the  First  IF  board.  The  ANOVA  results  indicated 
no  significant  (p<.05)  performance  effects  due  to  treatment  conditions 
or  proficiency  levels  on  probe  time. 

SECOND  IF  BOARD,  Tables  12  and  13  contain  the  ANOVA  data  for  number 
of  points  probed  on  the  Second  IF  board.  These  data  indicate  no 
significant  effects  due  to  treatment  condition  or  proficiency  level. 

The  ANOVA  results  on  probe  time  on  the  Second  IF  board  indicated 
a  significant  (p<.05)  effect  due  to  treatment  condition  (Tables  14  and 
15).  The  LSD  post  hoc  analysis  indicated  that  the  troubleshooting  CAI 
group  and  the  baseline  control  group  required  significantly  (p<.05) 
less  time  to  isolate  the  fault  than  the  control  (BASIC)  CAI  group 
(Table  16). 

CHI-SQUARE  ANALYSIS  -  TROUBLESHOOTING  SUCCESS 

One  measure  of  the  effectiveness  of  a  student's  troubleshooting 
performance  is  correct  fault  isolation  on  the  first  fault  diagnosis 
attempt.  If  the  training  effects  of  the  experimental  treatment  and 
control  conditions  were  equal,  then  we  would  expect  the  number  of 
students  who  had  correctly  diagnosed  the  fault  on  the  first  diagnosis 
attempt  to  be  equal  across  conditions.  Whether  or  not  a  student  was 
correct  on  the  first  fault  diagnosis  is  a  dichotomous  variable  which 
can  be  analyzed  using  a  Chi-Square  test.  This  is  a  comparison  of  a 
set  of  observed  frequencies  (number  correct  on  first  diagnosis)  with  a 
set  of  expected  frequencies  (expected  number  correct).  The  results  in 
this  section  examine  the  Chi-Square  tests  within  the  3  board  types. 

POWER  SUPPLY  BOARD.  Tables  17  and  18  contain  the  Chi-Square  frequency 
data  for  treatment  conditions  and  student  proficiency  levels.  As  the 
probabilities  indicate,  there  were  no  significant  frequency  differ¬ 
ences  between  the  independent  variables.  The  analysis  indicates  that 
the  assigned  treatment  condition  and  student  proficiency  level  had 
little  effect  on  troubleshooting  success.  The  success  rate  for  the 
experimental  condition  was  not  different  from  the  success  rate  of  the 
control  conditions. 

FIRST  IF  BOARD.  Table  19  contains  the  Chi-Square  data  by  treatment 
condition  and  indicates  no  significant  differences  in  success  rate  on 
the  first  fault  diagnosis  attempt.  Student  proficiency  level,  howev¬ 
er,  did  have  a  significant  (p<.01)  effect  on  whether  or  not  a  student 
had  a  correct  diagnosis  on  the  fault  diagnosis  (Table  20).  Further 
analysis  indicated  that  the  high  proficiency  students  had  significant¬ 
ly  (p<.01)  more  correct  first  attempts  than  incorrect,  significantly 
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TABLE  10.  FIRST 

IF  BOARD 

-  ANOVA  TOTALS 

- 

PROBE 

TIME  (MINUTES) 

PROFICIENCY  LEVEL 

TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

189 

250 

252 

Control  CAI 

111 

267 

288 

Control 

106 

193 

185 

TABLE  11. 

FIRST  IF  BOARD  - 

ANOVA  SUMMARY 

-  PROBE  TIME  (MINUTES) 

VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Proficiency 

3599.96 

2 

1799.98 

2.25 

Treatment 

1418.44 

2 

709.22 

.89 

Interaction 

715.24 

4 

178.81 

.22 

Error 

35920.80 

45 

798.24 

Total 

41654.29 

53 

NOTE:  No  Significant  Effects 
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TABLE  12.  SECOND  IF  BOARD  -  ANOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

166 

213 

180 

Control  CAI 

202 

245 

232 

Control 

116 

117 

181 

TABLE  13.  SECOND 

IF  BOARD  -  ANOVA 

SUMMARY  - 

NUMBER  OF  POINTS 

PROBED 

VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE 

F 

Prof ic iency 

234.38 

2 

117.19 

.28 

Treatment 

1958.12 

2 

979.06 

2.38 

Interact  ion 

478.88 

4 

119.72 

.29 

Error 

18438.65 

45 

411.97 

Total 

21210.07 

53 

NOTE:  No  Significant 

Effects 

21 
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TABLE  '4.  SECOND  IF  BOARD  -  ANOVA  TOTALS  - 
PROBE  TIME  (MINUTES) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

176 

154 

127 

Control  CAI 

198 

344 

204 

Control 

112 

168 

188 

TABLE  15.  SECOND  IF  BOARD  -  ANOVA  SUMMARY  -  PROBE  TIME  (MINUTES) 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Proficiency 

1020.30 

2 

510.15 

1.05 

Treatment 

2980.08 

2 

1490.04 

3.05* 

Interaction 

1972.92 

4 

493.23 

1.01 

Error 

21949.65 

45 

487.77 

Total 

27923.05 

53 

NOTE:  *p<.05 

TABLE  16.  SECOND  IF  BOARD  -  TREATMENT  CONDITION  - 


MEAN  PROBE 

TIME  (MINUTES) 

TREATMENT  CONDITION 

MEAN  TIME 

Troubleshooting  CAI 

25.39 

Control  CAI 

41.44 

Control 

26.00 
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TABLE  17.  POWER  SUPPLY  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  TREATMENT  CONDITION 

CORRECT 
FIRST  ATTEMPT 

TREATMENT  CONDITION  YES  NO 


Troubleshooting  CAI 

10 

8 

Control  CAI 

12 

6 

Control 

11 

7 

r-«. 

H 

<M 

p  =  .79 

TABLE  18.  POWER  SUPPLY  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  PROFICIENCY  LEVEL 


PROFICIENCY 

LEVEL 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


High 

13 

5 

Medium 

10 

8 

Low 

10 

8 

X2-  1.40 
p  ■  .50 
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TABLE  19.  FIRST  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  TREATMENT  CONDITION 

CORRECT 
FIRST  ATTEMPT 

TREATMENT  CONDITION  YES  NO 


Troubleshoot ing  CAI 


11  7 


Control  CAI  9  9 

Control  12  6 


X2-  1.07 
p  »  .59 


TABLE  20.  FIRST  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  PROFICIENCY  LEVEL 


PROFICIENCY 

LEVEL 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


High 

Medium 

Low 

X2-  9.36 

p  <.01 


15  3 

6  12 

11  7 
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(p<.03)  fewer  incorrect  first  attempts  than  the  medium  proficiency 
students,  and  significantly  (p<.05)  more  correct  first  attempts  than 
medium  proficiency  students. 

SECOND  IF  BOARD.  Tables  21  and  22  indicate  that  the  Chi-Square  anal¬ 
ysis  did  not  reveal  any  significant  (p<.05)  frequency  differences  due 
to  treatment  condition  or  proficiency  level.  The  number  of  students 
correct  and  incorrect  on  the  first  diagnosis  attempt  was  not  affected 
by  the  independent  variables. 

DESCRIPTIVE  ANALYSIS  -  TROUBLESHOOTING  LOGIC 

The  objective  of  this  portion  of  the  analysis  was  to  examine  the 
troubleshooting  logic  used  by  the  students,  assess  its  effectiveness, 
and  determine  the  effects  of  the  independent  variables  on  the  logic 
used.  In  order  to  determine  the  troubleshooting  strategy  or  logic 
used,  the  sequence  of  probes  taken  to  isolate  the  fault  was  analyzed. 
Each  strategy  used  was  recorded  and  classified  into  one  of  the  follow¬ 
ing  categories: 

a.  HALF-SPLIT  -  The  troubleshooter  successively  tests  the  mid¬ 
point  between  a  known  good  and  bad  signal  or  voltage  until 
the  fault  is  located. 

b.  LINEAR  I/O  -  The  troubleshooter  begins  at  the  board  input  and 
tests  the  voltage  or  signal  output  of  each  circuit  sequential¬ 
ly  until  the  faulty  circuit  or  stage  is  found. 

c.  LINEAR  TRACING  -  The  troubleshooter  begins  at  the  board  input 
and  tests  voltage  or  signals  sequentially  until  the  fault  is 
found. 

d.  LINEAR  IN-CIRCUIT  TRACING  -  After  a  particular  circuit  has 
been  isolated  by  any  method,  the  troubleshooter  tests  voltage 
or  signals  sequentially  within  the  specific  circuit  until  the 
fault  is  located. 

e.  RELIABILITY  TESTING  -  The  troubleshooter  successively  tests 
the  least  reliable  untested  component  until  the  fault  is 
found. 

f.  SYMPTOMATIC  -  The  troubleshooter  tests  the  circuit  area  or 
stage  whose  failure  would  cause  the  front  panel  symptoms 
observed. 

g.  RANDOM  -  The  troubleshooter  uses  no  logical  sequence  of 
tests . 
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TABLE  21.  SECOND  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  TREATMENT  CONDITION 

CORRECT 
FIRST  ATTEMPT 

TREATMENT  CONDITION  YES  NO 


11  7 

10  8 

10  8 

X2*  .15 
p  *  .  93 


Troubleshooting  CAI 
Control  CAI 
Control 


TABLE  22.  SECOND  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  PROFICIENCY  LEVEL 


PROFICIENCY 

LEVEL 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


High 

11 

7 

Medium 

10 

8 

Low 

10 

8 

X2-  .15 
p  -  .93 


1; 
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h.  LINEAR  COMPONENT  CHECKING  -  The  troubleshooter  sequentially 
tests  conductivity  of  components  until  a  fault  is  located. 

i.  OTHER  -  The  troubleshooter  appears  to  use  some  logical  tech¬ 
nique,  but  the  technique  cannot  be  identified. 

Many  times  the  sequence  of  probes  indicated  that  the  students  utilized 
more  than  one  technique  to  isolate  a  fault.  The  number  of  strategies 
used  and  the  sequence  of  strategies  were  recorded  until  the  student's 
first  attempt  to  classify  the  fault,  i.e.,  the  first  trip  to  the  L3 
with  a  fault  judgement.  This  section  decails  the  analysis  of  trouble¬ 
shooting  strategies  by  board  type. 

POWER  SUPPLY  BOARD.  The  Half-Split  technique  was  utilized  by  most 
students  as  their  first  strategy  (Table  23).  The  number  of  students 
using  Half-Split  first  is  significantly  (p<.01,  Chi-Square  test) 
greater  than  the  number  using  other  techniques  first  regardless  of 
treatment  condition.  After  using  the  Half-Split  technique,  students 
tended  to  try  different  techniques,  such  as  Linear  Component  Checking 
and  Random  (Table  24). 

Chi-Square  tests  were  used  to  determine  if  the  number  of  strat¬ 
egies  used  was  affected  by  the  independent  variables  or  if  the  number 
of  strategies  used  affected  troubleshooting  success.  Table  25  con¬ 
tains  the  Chi-Square  frequency  data,  by  treatment  conditions,  for  the 
number  of  students  using  1  to  4  (or  more)  strategies  prior  to  their 
first  trip  to  the  LS  with  a  fault  diagnosis.  Several  students  in  each 
condition  used  more  than  4  strategies,  but  the  majority  of  first  fault 
diagnosis  attempts  occurred  within  4  strategies.  The  analysis  indicat¬ 
ed  that  the  number  of  strategies  used  was  not  affected  by  treatment 
condition.  Table  26  contains  the  total  number  of  strategies  used 
prior  to  the  first  trip  to  the  LS  with  a  fault  judgement,  by  treatment 
condition  based  on  whether  the  first  attempt  resulted  in  a  correct  or 
incorrect  diagnosis.  The  Chi-Square  test  indicated  no  significant 
frequency  differences.  Thus,  the  number  of  strategies  used  was  not 
affected  by  treatment  conditions  and  did  not  affect  success  on  the 
first  attempt  at  a  diagnosis. 

Since  the  Half-Split  technique  was  the  predominant  method  used, 
and  the  method  under  study,  further  analysis  examined  whether  or  not 
students  who  used  it  first  were  more  successful  on  their  first  fault 
diagnosis  than  those  who  did  not  use  Half-Split.  The  data  contained 
in  Table  27  indicate  that  when  compared  to  all  other  strategies  com¬ 
bined,  the  Half-Split  technique  did  not  produce  more  students  correct 
on  the  first  attempt. 

FIRST  IF  BOARD.  The  Half-Split  technique  was  the  predominant  (p<.0i) 
first  strategy  used  on  the  First  IF  board,  regardless  of  treatment 
condition  (Table  28).  After  the  Half-Split,  students  tended  to  use 


FIRST  TROUBLESHOOTING  STRATEGY  USED  -  POWER  SUPPLY  BOARD 


TABLE  24.  SECOND,  THIRD  AND  FOURTH  STRATEGIES  USED  -  POWER  SUPPLY  BOARD 


NAVTRAEQUIPCEN  82-00119-1 


C/3 

-J 

H 

o 

Z 

at 

U3 

•—> 

Q 

Z 

X 

C 

u 

c 

C/3 

V 

C/3 

u: 

-J 

H 

H 

o 

Z 

< 

00 

U3 

0C 

H 

Q 

Cm 

z 

*— • 

=) 

C/3 

o 

< 

fern 

o 

CJ 

c/3 

X 

H 

CC 

CO 

o 

H 

u. 

z 

£ 

*>■4 

X 

CO 

< 

c— 

f— 

CJ 

C/3 

C/3 

-J 

o 

z 

as 

C*3 

H 

a 

Z 

X 

o 

H 

o 

03 

>» 

c/3 

U3 

-J 

C-l 

H 

o 

Z 

< 

30 

U3 

as 

H 

Q 

H 

Z 

NN 

X 

c/3 

o 

c 

H 

CJ 

CJ 

c/3 

O 

OS 

1— 1 

03 

X 

km 

H 

z 

s 

•-N 

3 

CO 

< 

H 

f- 

u 

CO 

—i  — 


©  —  iA 


' — .  60  C  u  Bt  --SC  w 

_>  e  *-1  c  —c  s 

•  3  *M  »-N  **N  8 

U  U  u  I-  y  fj  o  u  O 

<«  ion  n  is  n  <o  «  *j 

V  II  h  01  *H  In  -MO)  a 

e  C  H  C  O  H  -N  H  a 

■•n  <h  •  *  4)  >> 

-j  j  j  as  to 


o 

O  4J 

e 

u  01 
n  e  oo 

S  8..S 


Troubleshoot ing 
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TABLE  25.  POWER  SUPPLY  BOARD  -  NUMBER  OF  STUDENTS  USING 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

STUDENTS  USING 

ONE  TWO  THREE  FOUR 

TREATMENT  CONDITION  STRATEGY  STRATEGIES  STRATEGIES  STRATEGIES* 


Troubleshooting  CAI  64  3 


5 


Control  CAI  6615 

Control  7614 

X 2*  2.35 

p  *  .88 

*  Includes  students  using  more  than  four  strategies. 


TABLE  26.  POWER  SUPPLY  BOARD  -  SUCCESS  RATE  BASED  ON  TOTAL 
STRATEGIES  USED  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

TOTAL  STRATEGIES  USED 
CORRECT  INCORRECT 

TREATMENT  CONDITION  FIRST  ATTEMPT  FIRST  ATTEMPT 


Troubleshooting  CAI 

32 

16 

Control  CAI 

29 

16 

Control 

19 

19 

X2-  2.81 
p  “  .24 


TABLE  27.  POWER  SUPPLY  BOARD  -  SUCCESS  RATE 
OF  FIRST  DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 


FIRST  STRATEGY  USED 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


Half-Split 


28 


19 


All  Other  Strategies 


5  2 


X2-  .03 
p  -  .85 


30 
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Linear  I/O  as  their  second  strategy,  Linear  Component  Checking  as  the 
third  strategy,  with  no  predominant  fourth  strategy  (Table  29). 

\  Ch  i-Squar.?  test  was  used  to  determine  if  treatment  condition 
a  t  tec  t  r-d  the  number  of  strategies  a  student  used  prior  to  the  first 
trip  to  the  LS  with  a  fault  diagnosis.  Data  in  Table  30  indicate  that 
this  was  not  the  case.  The  total  number  of  strategies  used  by  ail 
students  in  each  condition  did  significantly  (p<.05)  affect  success  on 
the  first  trip  to  Che  LS  (Table  31).  Further  analysis  found  signif¬ 
icance  between  the  control  CAI  group  and  the  baseline  control  group  on 
incorrect  first  attempts,  i.e.,  the  students  in  the  baseline  control 
group  used  significantly  (p<.01)  fewer  strategies  on  incorrect  first 
attempts  than  the  control  CAI  group,  The  Half-Split  technique  did  not 
significantly  affect  success  on  the  first  diagnosis  attempt  when 
compared  to  all  other  techniques  (Table  32). 

SECOND  IF  BOARD.  The  first  troubleshooting  strategy  used  on  the 
Second  IF  board  was  predominantly  the  Half-Split  technique  (Table  33). 

The  first  technique  used  was  not  affected  by  treatment  condition.  The 
students  tended  to  use  Linear  Component  Checking  as  the  preferred 
second  strategy,  Half-Split  as  the  third  strategy,  with  no  preferred 
fourth  strategy  (Table  34). 

The  number  of  strategies  used  by  each  student,  prior  to  the  first 
attempted  diagnosis,  was  not  affected  by  treatment  condition  (Table  i 

35)  and  did  not  affect  the  success  rate  of  that  first  attempt  (Table 

36) .  The  students  using  Half-Split  as  their  first  strategy  were  not 
more  successful  than  students  using  all  the  other  strategies  combined 
(Table  37). 

ET  COURSE  COMPLETION  TIMES 

The  ET  Splice  course  is  self-paced,  and  one  of  the  hypothesized 
effects  of  the  troubleshooting  CAI  was  to  shorten  the  amount  of  time 
required  to  complete  the  subsequent  ET  Splice  curriculum.  Likewise, 
since  the  student  proficiency  level  was  based  on  the  completion  time 
of  the  prerequisite  course,  it  was  predicted  that  the  high  proficiency 
students  would  require  less  time  to  complete  the  course  than  medium 
and  low  proficiency  students  and  that  high  proficiency  students  receiv¬ 
ing  the  troubleshooting  CAI  would  require  less  time  than  all  other 
students . 

Tables  38  and  39  contain  the  ANOVA  totals  and  summary  data  for  ET 
course  completion  times  (in  total  course  hours).  The  results  indicate 
chat  both  proficiency  level  and  treatment  condition  had  a  significant 
(p<.05)  effect  on  ET  Splice  course  completion  time,  but  they  did  not 
have  a  significant  interaction.  The  mean  completion  times  for  stu¬ 
dents  classified  by  each  of  the  independent  variables  are  shown  in 
Tables  40  and  41.  The  LSD  post  hoc  analysis  indicated  that  high 
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TABLE  30.  FIRST  IF  BOARD  -  NUMBER  OF  STUDENTS  US  INC 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

STUDENTS  USING 

ONE  TWO  THREE  FOUR 

TREATMENT  CONDITION  STRATEGY  STRATEG IES  STRATEGIES  STRATEGIES* 


Troubleshooting  CAI  2484 

Control  CAI  17  6  4 


Control  3960 

X2=  7.30 
p  »  .29 

*  Includes  students  using  more  Chan  four  strategies. 


TABLE  31.  FIRST  IF  BOARD  -  SUCCESS  RATE  BASED  ON  TOTAL 
STRATEGIES  USED  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 


TOTAL  STRATEGIES  USED 

CORRECT 

INCORRECT 

TREATMENT  CONDITION 

FIRST  ATTEMPT 

FIRST  ATTEMPT 

Troubleshooting  CAl 

28 

29 

Control  CAI 

20 

32 

Control 

25 

14 

X2-  5.86 
p  -  .05 

TABLE  32.  FIRST  IF  BOARD  -  SUCCESS  RATE  OF  FIRST 
DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 

CORRECT 


FIRST 

ATTEMPT 

FIRST  STRATEGY  USED 

YES 

NO 

Haif-Splic 

30 

21 

All  Other  Strategies 

1 

1 

X2-  .23 
p  ■  .63 


34 


TABLE  34.  SECOND,  THIRD  AND  FOURTH  STRATEGIES  USED  -  SECOND  IF  BOARD 


*  TS  *  Troubleshooting 

**  The  total  number  of  students  is  less  because  not  all  students  utilized  more 
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TABLE  35.  SECOND  IF  BOARD  -  NUMBER  OF  STUDENTS  USING 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 


TREATMENT  CONDITION 

ONE 

STRATEGY 

STUDEN 

TWO 

STRATEGIES 

TS  USING 
THREE 

STRATEGIES 

FOUR 

STRATEGIES* 

Troubleshooting  CAI 

6 

5 

1 

6 

Control  CAI 

4 

9 

1 

4 

Control 

7 

8 

1 

;<2  =  4.00 

p  =  .68 

*  Includes  students 

using  more 

than  four  strategies. 

TABLE  36.  SECOND  IF  BOARD  -  SUCCESS  RATE  BASED  ON  TOTAL 
STRATEGIES  USED  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

TOTAL  STRATEGIES  USED 
CORRECT  INCORRECT 

TREATMENT  CONDITION  FIRST  ATTEMPT  FIRST  ATTEMPT 


Troubleshooting  CAI 
Control  CAI 
Control 


27 

18 

26 

20 

18 

15 

TABLE  37.  SECOND  IF  BOARD  -  SUCCESS  RATE  OF  FIRST 
DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 

CORRECT 


FIRST 

ATTEMPT 

FIRST  STRATEGY  USED 

YES 

NO 

Half-Split 

26 

19 

All  Other  Strategies 

<> 

5 

3 

X2-  .02 

p  *  .88 
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TABLE  38.  ET  COURSE  COMPLETION  TIME  - 
ANOVA  TOTALS  (HOURS) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

1693.70 

2255.50 

2873.40 

Control  CAI 

1676.70 

2381.70 

3231.10 

Control 

1582.20 

2171.10 

2702.10 

TABLE  39.  ET  COURSE  COMPLETION  TIME  -  ANOVA  SUMMARY 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Proficiency 

412774.00 

2 

206387.00 

94.72** 

Treatment 

19409.76 

2 

9704.88 

4. 45* 

Interaction 

9823.52 

4 

2455.88 

1.13 

Error 

98053.20 

45 

2178.96 

Total 

540059.40 

53 

**  p< .01 

*  p< .05 
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TABLE  40.  MEAN  COMPLETION  TIME  BY 
PROFICIENCY  LEVEL 


PROFICIENCY 

MEAN 

LEVEL 

TIME 

High 

275.14 

Med iura 

378.24 

Low 

489.26 

TABLE  41.  MEAN  COMPLETION  TIME  BY 
TREATMENT  CONDITION 


TREATMENT 

MEAN 

CONDITION 

TIME 

Troubleshooting  CAI 

379.03 

Control  CAI 

404.97 

Control 


358.63 
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proficiency  students  completed  the  course  in  significantly  fewer  hours 
than  the  medium  and  low  proficiency  students  and  that  the  medium 
proficiency  students  required  significantly  fewer  hours  than  the  low 
proficiency  students  ( p< .01).  The  LSD  post  hoc  analysis  for  treatment 
condition  indicated  that  the  baseline  control  group  completed  the 
course  in  significantly  less  time  than  the  group  receiving  the  control 
CAI  course  (p<.01).  There  was  no  significant  difference  in  course 
completion  time  between  the  students  receiving  the  troubleshooting  CAI 
course  and  those  receiving  no  treatment. 

STUDENT  COMMENTS 

The  comments  made  by  students  on  the  troubleshooting  CAI  question¬ 
naire  and  made  to  the  researcher  during  the  performance  tests  are 
summarized  and  outlined  in  the  following  2  sections. 

TROUBLESHOOTING  CAI  COMMENTS 

A  Troubleshooting  Strategy  Questionnaire  administered  to  the  18 
experimental  treatment  subjects,  subsequent  to  their  completion  of  the 
CAI,  indicated  the  following: 

a.  Fifteen  students  felt  the  CAI  unit  on  Isolation  Strategy  was 
the  most  valuable  in  helping  them  learn  effective 
troubleshooting. 

b.  CAI  units  on  System  Flow  Visualization,  System  Visualization 
and  Localization/Fixed  Flow  gave  students  the  least  amount  of 
difficulty. 

c.  All  18  experimental  subjects  felt  the  CAI  unit  on  Feedback 

was  the  most  difficult. 

d.  Of  the  three  media  used  in  presenting  the  experimental  treat¬ 
ment  condition,  all  18  subjects  preferred  computer  interac¬ 
tion;  whereas,  the  least  preferred  mode  of  instruction  was 

split  between  video  instruction  and  the  use  of  a  workbook. 

PERFORMANCE  TEST  COMMENTS.  The  researcher  recorded  pertinent  comments 
made  during  the  troubleshooting  performance  tests  and  the  predominant 
ones  are  as  follows: 

a.  Learn  more  in  the  testing  center  than  in  their  assigned 

class. 

b.  Do  not  understand  how  to  read  a  Simpson  Multimeter. 

c.  Prefer  using  a  digital  multimeter. 
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d.  Do  not  understand  how  to  set  up  the  equipment. 

e.  Do  not  remember  to  check  front  panel  symptoms  on  the  trainer. 

f.  Enjoyed  participating  in  qhe  research  performance  tests,  feel 
a  genuine  concern  regarding  their  performance. 

g.  Feel  confused  while  trying  to  locate  the  fault. 


NAVTRAEQUIPCEN  82-C-0119-1 


SECTION  IV 

CONCLUSIONS  AND  RECOMMENDATIONS 

If  Che  troubleshoot ing  CAI  used  in  this  research  had  enhanced  the 
BE&E  School's  training  curriculum,  then  the  students  receiving  that 
CAI  course  should  have  performed  significantly  better  than  the  other 
groups  on  the  performance  tests.  The  research  compared  treatment  and 
control  conditions  in  a  strict  experimental  environment.  Overall,  the 
results  indicate  that  the  troubleshooting  CAI  did  not  enhance  perfor¬ 
mance,  and  in  some  cases,  the  baseline  control  group  performed  signif¬ 
icantly  better  than  the  troubleshooting  CAI  group.  The  statistically 
significant  and  non-significant  results  indicate  that  the  control 
group  receiving  no  CAI  course  can  perform  as  well  as  or  better  than 
the  groups  receiving  the  CAI  treatments. 

If  the  troubleshooting  CAI  course  had  given  students  additional 
unique  instructional  material  to  utilize  Half-Split  more  efficiently, 
then  we  would  have  expected  students  receiving  the  troubleshooting  CAI 
to  use  Half-Split  more  than  the  other  groups.  However,  the  analysis 
indicated  that  all  groups  initially  used  the  Half-Split  technique  to 
localize  the  fault  and  that  the  troubleshooting  CAI  did  not  improve 
strategy  usage. 

It  should  be  noted  that  the  CAI  course  resulted  in  significantly 
improved  performance  during  the  pilot  study.  At  the  time  of  the  pilot 
study,  the  Half-Split  troubleshooting  technique  was  not  emphasized  in 
the  School.  In  the  time  period  between  completion  of  the  pilot  study 
and  initiation  of  the  full  study,  the  School  modified  the  curriculum 
toward  a  greater  emphasis  on  the  Half-Split  troubleshooting  technique. 
With  the  increased  emphasis  on  the  Half-Split  technique  in  the  reg¬ 
ular  curriculum,  the  CAI  course  on  Half-Split  did  not  lead  to  improved 
troubleshooting  performance.  In  fact,  the  significant  ANOVA  results 
indicated  negative  training  effects  on  the  Power  Supply  board  trouble¬ 
shooting  performance.  Of  the  3  boards,  the  Power  Supply  board  was  the 
only  one  with  feedback  loops.  All  students  receiving  the  troubleshoot¬ 
ing  CAI  course  felt  the  unit  on  feedback  was  the  most  difficult.  The 
CAI  instruction  could  have  interfered  with  their  troubleshooting 
efficiency  if  they  did  not  fully  comprehend  its  application. 

The  logic  analysis  and  review  of  the  performance  write-ups  indi¬ 
cate  that  students  use  essentially  the  same  strategies  regardless  of 
additional  training.  However,  review  of  the  logic  analysis  also 
indicated  that  while  students  may  sequentially  probe  points  in  a 
Half-Split  pattern,  they  do  not  always  probe  the  most  logical  points. 
The  Half-Split  technique  requires  the  user  to  probe,  take  a  reading, 
and  then  make  a  judgement  as  to  which  point  will  provide  information 
to  further  localize  the  fault.  Students  using  this  method  did  not 
always  correctly  interpret  the  information  they  were  gathering. 
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Missing  cues,  failure  to  recognize  faults,  and  the  inability  to  distin¬ 
guish  a  good  signal  from  a  bad  one,  were  recurring  problems.  Thus, 
the  Half-Split  technique,  by  itself,  did  not  always  lead  to  Che  isola¬ 
tion  of  the  fault.  This  finding  could  indicate  a  rote  probe  sequence 
rather  than  a  logical  usage  of  meter  readings  to  select  the  next 
appropriate  point  to  probe. 

The  treatment  control  condition  received  a  BASIC  program  which 
should  not  have  affected  performance  on  the  troubleshooting  casks. 
However,  data  trends  indicate  that  the  group  receiving  the  control  CAI 
performed  with  less  efficiency  than  the  other  groups.  Since  the  BE&E 
School  is  an  intensive  self-paced  program,  students  in  this  group  may 
have  been  affected  by  removal  from  the  electronic  training  environ¬ 
ment.  Removal  from  the  self-paced  program  should  have  affected  both 
CAI  groups,  but  the  control  CAI  may  have  had  additional  interference 
from  2-3  days  concentrated  work  on  an  unrelated  new  topic. 

The  hypothesized  effect  of  student  proficiency  level  was  support¬ 
ed  in  the  data  trends  and  significant  results.  As  defined  within  this 
BE&E  School  for  the  research,  high  proficiency  students  demonstrate 
more  efficient  troubleshooting  performance  than  medium  and  low  profi¬ 
ciency  students.  The  initial  proficiency  levels  set  at  the  beginning 
of  ET  Splice  can  be  used  to  predict  performance  during  the  curriculum 
and  to  predict  course  completion  times. 

In  general,  the  research  has  indicated  that: 

a.  The  off-the-shelf  troubleshooting  CAI,  as  used  in  this  re¬ 
search,  does  not  improve  student  troubleshooting  performance. 
The  CAI  and  School  curricula  should  be  reviewed  to  determine 
if  utilization  at  another  training  stage  will  enhance 
performance. 

b.  Review  of  the  curriculum  and  CAI  should  be  made  to  determine 
if  a  strategic  troubleshooting  course  developed  specifically 
for  BE&E  School  would  enhance  performance. 

c.  Review  of  curriculum  and  student  performance  should  be  made 
to  determine  if  students  understand  the  Half-Split  technique 
or  are  probing  points  by  rote  memory. 

d.  Student  proficiency  level  (based  on  BE&E  completion  times) 
can  be  used  to  predict  performance  in  ET  Splice  School.  Low 
proficiency  students  should  be  given  tutorial  assistance  to 
improve  their  troubleshooting  performance. 
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GLOSSARY 


AET 

Actual  Equipment  Trainer 

ANOVA 

Analysis  of  Variance 

BE&E 

Basic  Electricity  &  Electronics  School 

CAI 

Computer  Assisted  Instruction 

CM  I 

Computer  Managed  Instruction 

ET 

Electronic  Technician 

First 

IF 

First  Intermediate  Frequency  Board  (Medium 
Complexity) 

LS 

Learning  Supervisor 

LSD 

Least  Significant  Difference 

Power 

Supply 

Power  Supply  Board  (High  Complexity) 

Second 

IF 

Second  Intermediate  Frequency  Board  (Low 
Complexity) 
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